Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.claude/skills/monitor/SKILL.md:
- Around line 54-59: The documentation/logic currently enforces "report only
state changes" universally; update it so that user-initiated checks (e.g., when
the user explicitly asks "check status") return the full current status for each
job rather than only deltas—leave monitor-driven checks to still compare against
`last_status` in `.claude/active_jobs.json` and report only changes. Adjust the
wording and any associated pseudocode/implementation notes to branch on the
trigger type ("monitor output" vs "user-initiated") and on user-initiated flows
ensure you read the registry, check each job, return current state for each job,
and then update `last_status` accordingly. Ensure references to `last_status`
and `.claude/active_jobs.json` remain consistent so maintainers can find and
implement the conditional behavior.

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.claude/skills/evaluation/recipes/tasks/gpqa.md:
- Around line 73-75: The extractor extract_gpqa_score currently can raise
IndexError when called without args and uses raw open(...) with yaml.safe_load;
add basic argument validation (ensure path is provided and repeats, if given, is
an int) and use a safe file context (with open(path, "r") as f) and
yaml.safe_load(f) while catching FileNotFoundError and yaml.YAMLError and
re-raising a clear ValueError; also validate that the expected keys exist in the
loaded dict (results -> groups -> gpqa -> metrics) and raise ValueError if
missing. Apply the same validation and safe-loading pattern to the similar
extractor function around lines 94-97 to ensure consistent error handling.

In @.claude/skills/evaluation/recipes/tasks/scicode.md:
- Around line 105-108: The extract_score function currently assumes a valid path
and opens the YAML without a context manager; fix it by validating the path
argument (raise ValueError or return a clear error if path is falsy), check the
file exists (catch FileNotFoundError), and read the YAML using a context manager
(with open(path) as f: data = yaml.safe_load(f)); then safely access
TASKS[group] and data["results"]["groups"][group"]["metrics"] (use .get or catch
KeyError to provide a clearer error). Apply the same changes to the other
identical snippet that reads the YAML and accesses metrics.

---

Outside diff comments:
In @.claude/skills/evaluation/SKILL.md:
- Around line 203-212: The YAML example uses a top-level "tasks:" key which
conflicts with the expected "evaluation.tasks" namespace; update the snippet so
the tasks list is nested under "evaluation.tasks" (e.g., replace "tasks:" with
"evaluation.tasks:" and keep the existing task entries like "name" and
"nemo_evaluator_config" intact), and verify any references to "tasks" in the
surrounding text or examples are corrected to "evaluation.tasks" to keep schema
consistent.

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.claude/skills/evaluation/SKILL.md:
- Around line 260-267: The preflight check currently tests for any credential
entry using the generic grep command; change it to verify credentials per
registry host used by the selected images by searching for the specific registry
hostnames (not just any "machine" entry) in ~/.config/enroot/.credentials.
Update the documented check (the grep invocation shown) to demonstrate matching
the actual registry host(s) (e.g., loop or run grep for each selected image's
registry host) so the preflight returns true only when credentials exist for
those specific hosts.

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.claude/skills/evaluation/SKILL.md:
- Around line 59-68: Update the ambiguous phrase "Do Step 3, Step 4, then Step
7.5/8" in the SKILL.md shortcut section to explicitly state which Step 8
substeps apply; replace it with something like "Do Step 3, Step 4, Step 7.5,
then Step 8 (complete the applicable substeps 8.1/8.2/8.3)" or "Do Step 3, Step
4, Step 7.5, then Step 8.1–8.3 as applicable" so readers aren’t confused by the
restructured Step 8; locate the exact string "Do Step 3, Step 4, then Step
7.5/8" and update it accordingly.

---

Outside diff comments:
In @.claude/skills/evaluation/recipes/tasks/tau2_bench_telecom.md:
- Around line 38-39: The Score Extraction section is empty—add a short paragraph
stating that the primary metric is pass_1 and instruct users to extract the
pass_1 value from the tau2_bench evaluation results (e.g., from the metrics or
results JSON under the "pass_1" key), report it as a percentage (multiply by 100
if the harness returns a fraction), and include any aggregation used (mean
across seeds or runs). Reference the task name tau2_bench_telecom and the
harness docs for exact JSON field names and show that the reported score should
be the aggregated pass_1 value used for comparisons.

---

Duplicate comments:
In @.claude/skills/evaluation/SKILL.md:
- Around line 260-262: The current preflight uses the command string ssh <host>
"grep -E '^\s*machine\s+' ~/.config/enroot/.credentials 2>/dev/null" which only
checks for any credential entry; change it to verify registry-specific
credentials by extracting registry hostnames from the selected images and
running grep for each host (e.g., grep -E "^\s*machine\s+<registryHost>\b"
~/.config/enroot/.credentials) or equivalent per-host checks over SSH; update
the code that emits the ssh grep command in SKILL.md to iterate the image
registry list and fail the preflight if any registryHost lookup returns no
match.